fix: Make adaptive-crawler extra importable#2016
Conversation
vdusek
added
t-tooling
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2016 +/- ##
==========================================
+ Coverage 93.35% 93.39% +0.04%
==========================================
Files 179 179
Lines 12482 12482
==========================================
+ Hits 11652 11658 +6
+ Misses 830 824 -6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR fixes packaging for the adaptive-crawler optional extra so that pip install 'crawlee[adaptive-crawler]' can successfully import AdaptivePlaywrightCrawler by ensuring the extra also brings in the BeautifulSoup (bs4/html5lib) and Parsel dependencies that are imported at module import time.
Changes:
- Make
adaptive-crawlerdepend oncrawlee[beautifulsoup,parsel](self-referencing extras pattern already used byall). - Regenerate/update
uv.lockto reflect the new extra dependency graph and metadata.
Reviewed changes
Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
pyproject.toml |
Updates adaptive-crawler extra to include crawlee[beautifulsoup,parsel], ensuring transitive installation of bs4/parsel requirements. |
uv.lock |
Updates locked optional-dependency and requires-dist metadata so the lock reflects the new adaptive-crawler extra composition. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Problem
pip install 'crawlee[adaptive-crawler]'couldn't importAdaptivePlaywrightCrawler, because the module importsbs4andparselat the top level while the extra didn't include them.Changes
The
adaptive-crawlerextra now pulls incrawlee[beautifulsoup,parsel], using the same self-referencing pattern as theallextra.Verification
Installed only the
adaptive-crawlerextra into a clean venv, then importedAdaptivePlaywrightCrawlerand constructed both the BeautifulSoup and Parsel static-parser factories.